Imaging support device, imaging apparatus, imaging system, imaging support system, imaging support method, and program

ABSTRACT

An imaging support device that supports imaging performed by an imaging apparatus includes an acquisition portion that acquires an in-image shift amount between a predetermined position in a captured image obtained by capturing an imaging region by an imaging element and a position of a target subject image showing a target subject, and a focal length of the imaging apparatus, a derivation portion that derives a movement amount required for moving the position of the target subject image to a specific position by a position adjustment portion which adjusts the position of the target subject image in the captured image, based on the in-image shift amount acquired by the acquisition portion, the focal length acquired by the acquisition portion, and information related to a pixel interval of pixels in the imaging element, and an output portion that outputs the movement amount derived by the derivation portion.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of InternationalApplication No. PCT/JP2020/039425, filed Oct. 20, 2020, the disclosureof which is incorporated herein by reference in its entirety. Further,this application claims priority from Japanese Patent Application No.2019-196681, filed Oct. 29, 2019, the disclosure of which isincorporated herein by reference in its entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an imaging support device, an imagingapparatus, an imaging system, an imaging support system, an imagingsupport method, and a program.

2. Related Art

JP2017-215350A discloses an image shake correction apparatus comprisingan output unit that outputs a shake detection signal related to a shakeof the apparatus, a detection unit that detects a subject from acaptured image, a notification unit that notifies a capturing state, anda control unit that executes tracking of the subject and image shakecorrection based on a tracking target position of the detected subjectand the shake detection signal using a correction unit, in which thecontrol unit decides which of the tracking of the subject and the imageshake correction is to be preferentially executed in accordance with thecapturing state.

JP2017-126960A discloses an image shake correction apparatus comprisinga position detection unit that detects a position of a subject within ascreen, a speed detection unit that detects a speed of the subject usinga movement amount of the position of the subject within the screen, atracking amount calculation unit that calculates a tracking amount whichis necessary for moving the subject to a target position within thescreen and is a correction amount of a shift unit which moves a subjectimage on the screen, based on the position of the subject detectedimmediately before imaging and the speed of the subject in continuousimaging, and a control unit that drives the shift unit based on thetracking amount.

JP2017-063340A discloses an image shake correction apparatus thatcorrects an image shake based on a shake detection signal using a shakecorrection unit and comprises an acquisition unit which acquires subjectinformation related to a subject detected from a captured image, and acontrol unit that sets a reference position of the subject within acapturing angle of view based on the subject information and controlsthe shake correction unit such that the subject heads toward thereference position.

SUMMARY

One embodiment according to the disclosed technology provides an imagingsupport device, an imaging apparatus, an imaging system, an imagingsupport system, an imaging support method, and a program that cansupport tracking of a target subject in a case of capturing an imagingregion including the target subject.

A first aspect according to the disclosed technology is an imagingsupport device that supports imaging performed by an imaging apparatusincluding an imaging element and comprises an acquisition portion thatacquires an in-image shift amount between a predetermined position in acaptured image obtained by capturing an imaging region including atarget subject by the imaging element and a position of a target subjectimage showing the target subject, and a focal length of the imagingapparatus, a derivation portion that derives a movement amount requiredfor moving the position of the target subject image to a specificposition by a position adjustment portion which adjusts the position ofthe target subject image in the captured image, based on the in-imageshift amount acquired by the acquisition portion, the focal lengthacquired by the acquisition portion, and information related to a pixelinterval of pixels in the imaging element, and an output portion thatoutputs the movement amount derived by the derivation portion.

A second aspect according to the disclosed technology is the imagingsupport device according to the first aspect according to the disclosedtechnology, in which the position adjustment portion includes arevolution mechanism to which the imaging apparatus is attached and thatenables the imaging apparatus to revolve, and a shake correction portionthat corrects a shake which occurs due to a vibration exerted on theimaging apparatus, and the imaging support device further includes acontrol portion that performs an adjustment control of adjusting theposition of the target subject image in the captured image by operatingat least one of the revolution mechanism or the shake correction portionbased on the movement amount.

A third aspect according to the disclosed technology is the imagingsupport device according to the second aspect according to the disclosedtechnology, in which the movement amount is decided based on a firstmovement amount required for adjusting the position of the targetsubject image by the revolution mechanism and a second movement amountrequired for adjusting the position of the target subject image by theshake correction portion.

A fourth aspect according to the disclosed technology is the imagingsupport device according to the second or third aspect according to thedisclosed technology, in which the shake correction portion includes ashake correction element that is at least one of a lens for correctingthe shake by moving in accordance with the vibration or the imagingelement, and at the specific position, the shake correction element ispositioned at a center of a movable range of the shake correctionelement.

A fifth aspect according to the disclosed technology is the imagingsupport device according to the fourth aspect according to the disclosedtechnology, in which the acquisition portion further acquiressensitivity of the shake correction portion, and the derivation portionderives, based on a shake correction element shift amount between acenter position of the movable range and a current position of the shakecorrection element, and the sensitivity acquired by the acquisitionportion, a shake correction element movement amount required for movingthe current position to the center position as a second movement amountrequired for adjusting the position of the target subject image by theshake correction portion.

A sixth aspect according to the disclosed technology is the imagingsupport device according to any one of the third to fifth aspectsaccording to the disclosed technology, in which the first movementamount is decided based on a value obtained by dividing a product of thein-image shift amount and the pixel interval by the focal length.

A seventh aspect according to the disclosed technology is the imagingsupport device according to any one of the third to sixth aspectsaccording to the disclosed technology, in which the movement amount isobtained by combining the first movement amount and the second movementamount derived by the derivation portion.

An eighth aspect according to the disclosed technology is the imagingsupport device according to any one of the second to seventh aspectsaccording to the disclosed technology, in which the control portionperforms correction of the shake by the shake correction portion and theadjustment control in a time-division manner.

A ninth aspect according to the disclosed technology is the imagingsupport device according to the eighth aspect according to the disclosedtechnology, in which the control portion causes the shake correctionportion to perform the correction of the shake while the imagingapparatus is revolving by the revolution mechanism, and performs theadjustment control while the revolution of the imaging apparatus by therevolution mechanism is stopped.

A tenth aspect according to the disclosed technology is the imagingsupport device according to the eighth or ninth aspect according to thedisclosed technology, in which the adjustment control is a control ofadjusting the position of the target subject image by the shakecorrection portion after the position of the target subject image isadjusted by the revolution mechanism.

An eleventh aspect according to the disclosed technology is the imagingsupport device according to any one of the second to tenth aspectsaccording to the disclosed technology, in which the revolution mechanismis a 2-axis revolution mechanism that enables the imaging apparatus torevolve in a first direction and a second direction which intersectswith the first direction, and the shake correction portion is at leastone of an optical shake correction mechanism or an electronic shakecorrection portion.

A twelfth aspect according to the disclosed technology is the imagingsupport device according to the eleventh aspect according to thedisclosed technology, in which the optical shake correction mechanism isat least one of a lens moving type shake correction mechanism or animaging element moving type shake correction mechanism.

A thirteenth aspect according to the disclosed technology is the imagingsupport device according to any one of the first to twelfth aspectsaccording to the disclosed technology, in which the acquisition portionfurther acquires the information related to the pixel interval.

A fourteenth aspect according to the disclosed technology is the imagingsupport device according to any one of the first to thirteenth aspectsaccording to the disclosed technology, in which the output portionoutputs the movement amount to an outside.

A fifteenth aspect according to the disclosed technology is the imagingsupport device according to any one of the first to fourteenth aspectsaccording to the disclosed technology, in which the movement amount isdecided based on a movement velocity of the target subject in a casewhere the target subject is moving.

A sixteenth aspect according to the disclosed technology is the imagingsupport device according to the fifteenth aspect according to thedisclosed technology, in which the movement velocity includes aplurality of velocities obtained by decomposing the movement velocityinto a plurality of different directions.

A seventeenth aspect according to the disclosed technology is an imagingapparatus comprising the imaging support device according to any one ofthe first to sixteenth aspects according to the disclosed technology,and the imaging element, in which the imaging support device supportsimaging for the imaging element.

An eighteenth aspect according to the disclosed technology is an imagingsystem comprising the imaging apparatus according to the seventeenthaspect according to the disclosed technology, and a control device thatperforms at least one of a control of displaying an image on which anadjustment result of the position of the target subject image isreflected based on the movement amount derived by the derivation portionon a display portion, or a control of storing image data indicating theimage on which the adjustment result is reflected in a storage portion.

A nineteenth aspect according to the disclosed technology is an imagingsystem comprising an imaging support device that supports imagingperformed by an imaging apparatus including an imaging element, and theimaging element, in which the imaging support device includes anacquisition portion that acquires an in-image shift amount between apredetermined position in a captured image obtained by capturing animaging region including a target subject by the imaging element and aposition of a target subject image showing the target subject, and afocal length of the imaging apparatus, a derivation portion that derivesa movement amount required for moving the position of the target subjectimage to a specific position by a position adjustment portion whichadjusts the position of the target subject image in the captured image,based on the in-image shift amount acquired by the acquisition portion,the focal length acquired by the acquisition portion, and informationrelated to a pixel interval of pixels in the imaging element, and anoutput portion that outputs the movement amount derived by thederivation portion, the position adjustment portion includes arevolution mechanism to which the imaging apparatus is attached and thatenables the imaging apparatus to revolve, and a shake correction portionthat corrects a shake which occurs due to a vibration exerted on theimaging apparatus, and the imaging support device further includes acontrol portion that performs an adjustment control of adjusting theposition of the target subject image in the captured image by operatingat least one of the revolution mechanism or the shake correction portionbased on the movement amount.

A twentieth aspect according to the disclosed technology is an imagingsupport system comprising the imaging support device according to anyone of the first to sixteenth aspects according to the disclosedtechnology, and the position adjustment portion, in which the derivationportion included in the imaging support device derives the movementamount.

A twenty-first aspect according to the disclosed technology is animaging support method of supporting imaging performed by an imagingapparatus including an imaging element, the imaging support methodcomprising acquiring an in-image shift amount between a predeterminedposition in a captured image obtained by capturing an imaging regionincluding a target subject by the imaging element and a position of atarget subject image showing the target subject, and a focal length,deriving a movement amount required for moving the position of thetarget subject image to a specific position by a position adjustmentportion which adjusts the position of the target subject image in thecaptured image, based on the acquired in-image shift amount, theacquired focal length, and a pixel interval of pixels of the imagingelement, and outputting the derived movement amount.

A twenty-second aspect according to the disclosed technology is aprogram causing a computer to execute a process of supporting imagingperformed by an imaging apparatus including an imaging element, theprocess comprising acquiring an in-image shift amount between apredetermined position in a captured image obtained by capturing animaging region including a target subject by the imaging element and aposition of a target subject image showing the target subject, and afocal length, deriving a movement amount required for moving theposition of the target subject image to a specific position by aposition adjustment portion which adjusts the position of the targetsubject image in the captured image, based on the acquired in-imageshift amount, the acquired focal length, and a pixel interval of pixelsof the imaging element, and outputting the derived movement amount.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the technology of the disclosure will bedescribed in detail based on the following figures, wherein:

FIG. 1 is a schematic configuration diagram illustrating an example of aconfiguration of a surveillance system according to an embodiment;

FIG. 2 is a perspective view illustrating an example of an exterior of asurveillance camera according to the embodiment;

FIG. 3 is a perspective view illustrating an example of the exterior ofthe surveillance camera according to the embodiment;

FIG. 4 is a block diagram illustrating an example of a configuration ofan optical system and an electric system of the surveillance cameraaccording to the embodiment;

FIG. 5 is a block diagram illustrating an example of a configuration ofan electric system of a management apparatus and a revolution mechanismaccording to the embodiment;

FIG. 6 is a function block diagram illustrating an example of functionsof a CPU included in the surveillance camera according to theembodiment;

FIG. 7 is a conceptual diagram illustrating an example of a capturedimage according to the embodiment;

FIG. 8 is a conceptual diagram for describing derivation of a firstmovement amount according to the embodiment;

FIG. 9 is a conceptual diagram for describing a movable range of a shakecorrection element according to the embodiment;

FIG. 10 is a conceptual diagram for describing the movable range of theshake correction element according to the embodiment;

FIG. 11 is a conceptual diagram illustrating a screen display exampledisplayed by a display portion according to the embodiment;

FIG. 12 is a conceptual diagram for describing position adjustmentprocessing according to the embodiment;

FIG. 13 is a flowchart illustrating an example of a flow of positionadjustment processing according to the embodiment;

FIG. 14 is a conceptual diagram illustrating an example of a displayportion of the surveillance camera according to the embodiment;

FIG. 15 is a function block diagram illustrating a modification exampleof the functions of the CPU included in the surveillance cameraaccording to the embodiment;

FIG. 16 is a schematic configuration diagram illustrating an example ofa configuration of the surveillance camera according to the embodiment;

FIG. 17 is a conceptual diagram illustrating a screen display exampledisplayed by the display portion according to the embodiment;

FIG. 18 is a function block diagram illustrating a modification exampleof the functions of the CPU included in the surveillance cameraaccording to the embodiment;

FIG. 19 is a conceptual diagram for describing the position adjustmentprocessing according to the embodiment;

FIG. 20 is a conceptual diagram for describing the position adjustmentprocessing according to the embodiment;

FIG. 21A is a flowchart illustrating an example of the flow of positionadjustment processing according to the embodiment;

FIG. 21B is a flowchart illustrating an example of a flow of velocitydetermination processing according to the embodiment; and

FIG. 22 is a conceptual diagram illustrating an example of an aspect inwhich a position adjustment program is installed on a computer in thesurveillance camera from a storage medium storing the positionadjustment program according to the embodiment.

DETAILED DESCRIPTION

An example of an embodiment according to the disclosed technology willbe described in accordance with the appended drawings.

First, words used in the following description will be described.

CPU is an abbreviation for “Central Processing Unit”. GPU is anabbreviation for “Graphics Processing Unit”. ASIC is an abbreviation for“Application Specific Integrated Circuit”. PLD is an abbreviation for“Programmable Logic Device”. FPGA is an abbreviation for“Field-Programmable Gate Array”. AFE is an abbreviation for “AnalogFront End”. DSP is an abbreviation for “Digital Signal Processor”. SoCis an abbreviation for “System-on-a-chip”. SSD is an abbreviation for“Solid State Drive”. USB is an abbreviation for “Universal Serial Bus”.HDD is an abbreviation for “Hard Disk Drive”. EEPROM is an abbreviationfor “Electrically Erasable and Programmable Read Only Memory”. EL is anabbreviation for “Electro-Luminescence”. A/D is an abbreviation for“Analog/Digital”. I/F is an abbreviation for “Interface”. UI is anabbreviation for “User Interface”. WAN is an abbreviation for “Wide AreaNetwork”. ISP is an abbreviation for “Image Signal Processor”. CMOS isan abbreviation for “Complementary Metal Oxide Semiconductor”. CCD is anabbreviation for “Charge Coupled Device”. SWIR is an abbreviation for“Short-Wavelength Infrared”.

In the description of the present specification, “vertical” refers tobeing vertical in a sense of not only being completely vertical but alsoincluding an error generally allowed in the technical field to which thedisclosed technology belongs. In the description of the presentspecification, “horizontal” refers to being horizontal in a sense of notonly being completely horizontal but also including an error generallyallowed in the technical field to which the disclosed technologybelongs. In the description of the present specification, “parallel”refers to being parallel in a sense of not only being completelyparallel but also including an error generally allowed in the technicalfield to which the disclosed technology belongs. In the description ofthe present specification, “perpendicular” refers to being perpendicularin a sense of not only being completely perpendicular but also includingan error generally allowed in the technical field to which the disclosedtechnology belongs. In the description of the present specification,“same” refers to being the same in a sense of not only being completelythe same but also including an error generally allowed in the technicalfield to which the disclosed technology belongs.

First Embodiment

As illustrated in FIG. 1 as an example, a surveillance system 2comprises a surveillance camera 10 and a management apparatus 11. Thesurveillance system 2 is an example of an “imaging system” and an“imaging support system” according to the embodiment of the disclosedtechnology. The surveillance camera 10 is an example of an “imagingapparatus” according to the embodiment of the disclosed technology.

The surveillance camera 10 is installed in an indoor or outdoor post orwall, a part (for example, a rooftop) of a building, or the like througha revolution mechanism 16, described later, images a surveillance targetthat is a subject, and generates a motion picture image by the imaging.The motion picture image includes images of a plurality of framesobtained by imaging. The surveillance camera 10 transmits the motionpicture image obtained by imaging to the management apparatus 11 througha communication line 12.

The management apparatus 11 comprises a display 13 and a secondarystorage device 14. Examples of the display 13 include a liquid crystaldisplay or an organic EL display. The display 13 is an example of a“display portion (display)” according to the embodiment of the disclosedtechnology.

Examples of the secondary storage device 14 include an HDD. Thesecondary storage device 14 may be a non-volatile memory such as a flashmemory, an SSD, or an EEPROM instead of the HDD. The secondary storagedevice 14 is an example of a “storage portion (storage device)”according to the embodiment of the disclosed technology.

In the management apparatus 11, the motion picture image transmitted bythe surveillance camera 10 is received, and the received motion pictureimage is displayed on the display 13 or stored in the secondary storagedevice 14.

The surveillance camera 10 is attached to the revolution mechanism 16.The revolution mechanism 16 enables the surveillance camera 10 torevolve. Specifically, the revolution mechanism 16 is a 2-axisrevolution mechanism that enables the surveillance camera 10 to revolvein a first direction and a second direction intersecting with the firstdirection. As illustrated in FIG. 2 as an example, the revolutionmechanism 16 enables the surveillance camera 10 to revolve in arevolution direction (hereinafter referred to as a “pitch direction”) ofwhich a central axis is a pitch axis PA. In addition, as illustrated inFIG. 3 as an example, the revolution mechanism 16 enables thesurveillance camera 10 to revolve in a revolution direction (hereinafterreferred to as a “yaw direction”) of which a central axis is a yaw axisYA. The revolution mechanism 16 is an example of a “revolutionmechanism” according to the embodiment of the disclosed technology. Inaddition, the “pitch direction” is an example of a “first direction”according to the embodiment of the disclosed technology, and the yawdirection is an example of a “second direction” according to theembodiment of the disclosed technology. In the present embodiment, whilethe 2-axis revolution mechanism is illustrated as the revolutionmechanism 16, the disclosed technology is not limited thereto. Thedisclosed technology is also established in a case where a 3-axisrevolution mechanism is applied.

As illustrated in FIG. 4 as an example, the surveillance camera 10comprises an optical system 15 and an imaging element 25. The imagingelement 25 is positioned on a rear stage of the optical system 15. Theoptical system 15 comprises an objective lens 15A and a lens group 15B.The objective lens 15A and the lens group 15B are arranged in an orderof the objective lens 15A and the lens group 15B along an optical axisOA of the optical system 15 from the surveillance target side (objectside) to a light-receiving surface 25A side (image side) of the imagingelement 25. The lens group 15B includes a zoom lens 15B2 and the like.The zoom lens 15B2 is movably supported along the optical axis OA by amoving mechanism 21. The moving mechanism 21 moves the zoom lens 15B2along the optical axis OA in accordance with motive power provided froma motor (not illustrated) for the zoom lens. In addition, the lens group15B includes an anti-vibration lens 15B1. The anti-vibration lens 15B1changes in a direction perpendicular to an optical axis of theanti-vibration lens 15B1 in accordance with the provided motive power.

By the optical system 15 configured in such a manner, an image ofsurveillance target light showing the surveillance target is formed onthe light-receiving surface 25A. The imaging element 25 is an example ofan “imaging element” according to the embodiment of the disclosedtechnology.

Examples of a vibration exerted on the surveillance camera 10 include,in a case of an outdoor space, a vibration caused by traffic of anautomobile, a vibration caused by wind, a vibration caused byconstruction work, and the like and, in a case of an indoor space, avibration caused by an operation of an air conditioner, a vibrationcaused by entrance and exit of a person, and the like. In addition,examples of the vibration exerted on the surveillance camera 10 includea vibration during revolution of the surveillance camera 10 by therevolution mechanism 16, a vibration in a case where a revolutionoperation performed by the revolution mechanism 16 is started orstopped, and the like. Thus, in the surveillance camera 10, a shakeoccurs due to the vibration exerted on the surveillance camera 10(hereinafter, simply referred to as the “vibration”).

In the present embodiment, the “shake” refers to a phenomenon in which asubject image on the light-receiving surface 25A changes due to a changein positional relationship between the optical axis OA and thelight-receiving surface 25A in the surveillance camera 10. In otherwords, the “shake” is said to be a phenomenon in which an optical imageobtained by forming the image on the light-receiving surface 25A changesby inclination of the optical axis OA due to the vibration exerted onthe surveillance camera 10. For example, changing of the optical axis OAmeans inclination of the optical axis OA with respect to a referenceaxis (for example, the optical axis OA before the shake occurs).Hereinafter, the shake that occurs due to the vibration will be simplyreferred to as the “shake”.

Therefore, the surveillance camera 10 comprises a shake correctionportion 51. The shake correction portion 51 is an example of a “shakecorrection component” according to the embodiment of the disclosedtechnology. The shake correction portion 51 includes a mechanical shakecorrection portion 29 and an electronic shake correction portion 33. Theshake correction portion 51 corrects the shake. The mechanical shakecorrection portion 29 is an example of an “optical shake correctionmechanism” according to the embodiment of the disclosed technology. Themechanical shake correction portion 29 is a mechanism that corrects theshake by applying motive power generated by a driving source such as amotor (for example, a voice coil motor) to the anti-vibration lens tomove the anti-vibration lens in a direction perpendicular to an opticalaxis of an imaging optical system. The electronic shake correctionportion 33 corrects the shake by performing image processing on acaptured image based on a shake amount. That is, the shake correctionportion 51 mechanically or electronically corrects the shake using ahardware configuration and/or a software configuration. Here, mechanicalcorrection of the shake refers to correction of the shake implemented bymechanically moving a shake correction element such as an anti-vibrationlens and/or an imaging element using motive power generated by a drivingsource such as a motor (for example, a voice coil motor). Electroniccorrection of the shake refers to correction of the shake implemented byperforming the image processing by a processor. In the presentembodiment, “correction of the shake” includes a meaning of removing theshake and also a meaning of reducing the shake.

The mechanical shake correction portion 29 comprises the anti-vibrationlens 15B1, an actuator 17, a driver 23, and a position detection sensor39.

Various well-known methods can be employed as a method of correcting theshake by the mechanical shake correction portion 29. In the presentembodiment, a method of correcting the shake by moving theanti-vibration lens 15B1 based on the shake amount detected by a shakeamount detection sensor 40 (described later) is employed as the methodof correcting the shake. Specifically, the shake is corrected by movingthe anti-vibration lens 15B1 in a direction of canceling the shake by anamount with which the shake is canceled.

The actuator 17 is attached to the anti-vibration lens 15B1. Theactuator 17 is a shift mechanism in which a voice coil motor is mounted,and changes the anti-vibration lens 15B1 in the direction perpendicularto the optical axis of the anti-vibration lens 15B1 by driving the voicecoil motor. Here, while the shift mechanism in which the voice coilmotor is mounted is employed as the actuator 17, the disclosedtechnology is not limited thereto. Other motive power sources such as astepping motor or a piezo element may be applied instead of the voicecoil motor.

The actuator 17 is controlled by the driver 23. Driving the actuator 17under control of the driver 23 mechanically changes a position of theanti-vibration lens 15B1 with respect to the optical axis OA.

The position detection sensor 39 detects the current position of theanti-vibration lens 15B1 and outputs a position signal indicating thedetected current position. Here, a device including a hall element isemployed as an example of the position detection sensor 39. Here, thecurrent position of the anti-vibration lens 15B1 refers to the currentposition in a two-dimensional plane of the anti-vibration lens. Thetwo-dimensional plane of the anti-vibration lens refers to atwo-dimensional plane perpendicular to the optical axis of theanti-vibration lens 15B1. In the present embodiment, while the deviceincluding the hall element is employed as an example of the positiondetection sensor 39, the disclosed technology is not limited thereto. Amagnetic sensor, a photosensor, or the like may be employed instead ofthe hall element.

The surveillance camera 10 comprises a computer 19, a DSP 31, an imagememory 32, the electronic shake correction portion 33, a communicationI/F 34, the shake amount detection sensor 40, and a UI system device 43.The computer 19 comprises a memory 35, a storage 36, and a CPU 37. Theelectronic shake correction portion 33 is an example of an “electronicshake correction portion” according to the embodiment of the disclosedtechnology. In addition, the CPU 37 is an example of an “imaging supportdevice” according to the embodiment of the disclosed technology.

The imaging element 25, the DSP 31, the image memory 32, the electronicshake correction portion 33, the communication I/F 34, the memory 35,the storage 36, the CPU 37, the shake amount detection sensor 40, andthe UI system device 43 are connected to a bus 38. In addition, thedriver 23 is connected to the bus 38. In the example illustrated in FIG.4 , while one bus is illustrated as the bus 38 for convenience ofillustration, a plurality of buses may be used. The bus 38 may be aserial bus or may be a parallel bus including a data bus, an addressbus, a control bus, and the like.

The memory 35 temporarily stores various information and is used as awork memory. Examples of the memory 35 include a RAM. However, thedisclosed technology is not limited thereto. A storage device of othertypes may be used. The storage 36 is a non-volatile storage device.Here, a flash memory is employed as an example of the storage 36. Theflash memory is merely an example. Examples of the storage 36 includevarious non-volatile memories such as a magnetoresistive memory and/or aferroelectric memory instead of the flash memory or together with theflash memory. In addition, the non-volatile storage device may be anEEPROM, an HDD, and/or an SSD or the like. The storage 36 stores variousprograms for the surveillance camera 10. The CPU 37 controls the entiresurveillance camera 10 by reading out various programs from the storage36 and executing the read various programs on the memory 35.

The imaging element 25 is a CMOS image sensor. The imaging element 25images the surveillance target at a predetermined frame rate under aninstruction of the CPU 37. Here, for example, the “predetermined framerate” refers to a few tens of frames/second to a few hundredframes/second. The imaging element 25 may incorporate a control device(imaging element control device). In this case, the imaging elementcontrol device performs detailed controls inside imaging element 25 inaccordance with an imaging instruction output by the CPU 37. Inaddition, the imaging element 25 may image a target subject at thepredetermined frame rate under an instruction of the DSP 31. In thiscase, the imaging element control device performs the detailed controlsinside the imaging element 25 in accordance with the imaging instructionoutput by the DSP 31. The DSP 31 may be referred to as an ISP.

The light-receiving surface 25A is formed with a plurality ofphotosensitive pixels (not illustrated) arranged in a matrix. In theimaging element 25, photoelectric conversion is performed for eachphotosensitive pixel by exposing each photosensitive pixel. Chargesobtained by performing the photoelectric conversion for eachphotosensitive pixel correspond to an analog imaging signal indicatingthe surveillance target. Here, a plurality of photoelectric conversionelements (for example, photoelectric conversion elements in which colorfilters are arranged) having sensitivity to visible light are employedas the plurality of photosensitive pixels. In the imaging element 25, aphotoelectric conversion element having sensitivity to light of red (R)(for example, a photoelectric conversion element in which an R filtercorresponding to R is arranged), a photoelectric conversion elementhaving sensitivity to light of green (G) (for example, a photoelectricconversion element in which a G filter corresponding to G is arranged),and a photoelectric conversion element having sensitivity to light ofblue (B) (for example, a photoelectric conversion element in which a Bfilter corresponding to B is arranged) are employed as the plurality ofphotoelectric conversion elements. In the surveillance camera 10,imaging based on the visible light (for example, light on a shortwavelength side of less than or equal to approximately 700 nanometers)is performed using these photosensitive pixels. However, the presentembodiment is not limited thereto. Imaging based on infrared light (forexample, light on a long wavelength side of greater than approximately700 nanometers) may be performed. In this case, a plurality ofphotoelectric conversion elements having sensitivity to the infraredlight may be used as the plurality of photosensitive pixels.Particularly, for example, an InGaAs sensor and/or a type-2 quantum well(T2SL; Simulation of Type-II Quantum Well) sensor may be used forimaging for SWIR.

The imaging element 25 generates a digital image that is a digitalimaging signal by performing signal processing such as A/D conversion onthe analog imaging signal. The imaging element 25 is connected to theDSP 31 through the bus 38 and outputs the generated digital image to theDSP 31 in units of frames through the bus 38. Here, the digital image isan example of a “captured image” according to the embodiment of thedisclosed technology.

Here, while the CMOS image sensor is illustratively described as anexample of the imaging element 25, the disclosed technology is notlimited thereto. A CCD image sensor may be applied as the imagingelement 25. In this case, the imaging element 25 is connected to the bus38 through an AFE (not illustrated) that incorporates a CCD driver. TheAFE generates the digital image by performing the signal processing suchas the A/D conversion on the analog imaging signal obtained by theimaging element 25 and outputs the generated digital image to the DSP31. The CCD image sensor is driven by the CCD driver incorporated in theAFE. The CCD driver may be independently provided.

The DSP 31 performs various digital signal processing on the digitalimage. For example, the various digital signal processing refers todemosaicing, noise removal processing, gradation correction processing,and color correction processing.

The DSP 31 outputs the digital image after the digital signal processingto the image memory 32 for each frame. The image memory 32 stores thedigital image from the DSP 31. Hereinafter, for convenience ofdescription, the digital image stored in the image memory 32 will bereferred to as the “captured image”.

The shake amount detection sensor 40 is, for example, a device includinga gyro sensor and detects the shake amount of the surveillance camera10. In other words, the shake amount detection sensor 40 detects theshake amount for each of a pair of axial directions. The gyro sensordetects an amount of a rotational shake about each axis (refer to FIG. 1) of the pitch axis PA, the yaw axis YA, and a roll axis RA (axisparallel to the optical axis OA). The shake amount detection sensor 40detects the shake amount of the surveillance camera 10 by converting theamount of the rotational shake about the pitch axis PA and the amount ofthe rotational shake about the yaw axis YA detected by the gyro sensorinto a shake amount in a two-dimensional plane parallel to the pitchaxis PA and the yaw axis YA.

Here, while the gyro sensor is illustrated as an example of the shakeamount detection sensor 40, this is merely an example. The shake amountdetection sensor 40 may be an acceleration sensor. The accelerationsensor detects the shake amount in the two-dimensional plane parallel tothe pitch axis PA and the yaw axis YA. The shake amount detection sensor40 outputs the detected shake amount to the CPU 37.

In addition, here, while an example of a form of detecting the shakeamount by the shake amount detection sensor 40 that is a physical sensoris illustrated, the disclosed technology is not limited thereto. Forexample, a movement vector obtained by comparing captured images thatare stored in the image memory 32 and are adjacent to each other in timeseries may be used as the shake amount. In addition, a finally usedshake amount may be derived based on the shake amount detected by thephysical sensor and the movement vector obtained by the imageprocessing.

The CPU 37 acquires the shake amount detected by the shake amountdetection sensor 40 and controls the mechanical shake correction portion29 and the electronic shake correction portion 33 based on the acquiredshake amount. The shake amount detected by the shake amount detectionsensor 40 is used for correction of the shake by each of the mechanicalshake correction portion 29 and the electronic shake correction portion33. The mechanical shake correction portion 29 and the electronic shakecorrection portion 33 correct the shake in accordance with the shakeamount detected by the shake amount detection sensor 40.

The electronic shake correction portion 33 is a device including anASIC. The electronic shake correction portion 33 corrects the shake byperforming the image processing on the captured image in the imagememory 32 based on the shake amount detected by the shake amountdetection sensor 40.

Here, while the device including the ASIC is illustrated as theelectronic shake correction portion 33, the disclosed technology is notlimited thereto. For example, a device including an FPGA or a PLD may beused. In addition, for example, the electronic shake correction portion33 may be a device including a plurality out of the ASIC, the FPGA, andthe PLD. In addition, a computer including a CPU, a storage, and amemory may be employed as the electronic shake correction portion 33.The number of CPUs may be singular or plural. In addition, theelectronic shake correction portion 33 may be implemented by acombination of a hardware configuration and a software configuration.

The communication I/F 34 is, for example, a network interface andcontrols transfer of various information with respect to the managementapparatus 11 through a network. Examples of the network include a WANsuch as the Internet or a public communication network. Communicationbetween the surveillance camera 10 and the management apparatus 11 iscontrolled.

The UI system device 43 comprises a reception device 43A and a display43B. For example, the reception device 43A includes a hard key and atouch panel and receives various instructions from a user or the like ofthe surveillance system 2 (hereinafter, simply referred to as the “useror the like”). The CPU 37 acquires the various instructions received bythe reception device 43A and operates in accordance with the acquiredinstructions.

The display 43B displays various information under control of the CPU37. Examples of the various information displayed on the display 43Binclude contents of the various instructions received by the receptiondevice 43A and the captured image.

As illustrated in FIG. 5 as an example, the revolution mechanism 16comprises a yaw axis revolution mechanism 71, a pitch axis revolutionmechanism 72, a motor 73, a motor 74, a driver 75, and a driver 76. Theyaw axis revolution mechanism 71 causes the surveillance camera 10 torevolve in the yaw direction. The motor 73 is driven to generate motivepower under control of the driver 75. The yaw axis revolution mechanism71 causes the surveillance camera 10 to revolve in the yaw direction byreceiving the motive power generated by the motor 73. The motor 74 isdriven to generate motive power under control of the driver 76. Thepitch axis revolution mechanism 72 causes the surveillance camera 10 torevolve in the pitch direction by receiving the motive power generatedby the motor 74.

As illustrated in FIG. 5 as an example, the management apparatus 11comprises the display 13, the secondary storage device 14, a controldevice 60, a reception device 62, and communication I/Fs 66 to 68. Thecontrol device 60 comprises a CPU 60A, a storage 60B, and a memory 60C.Each of the reception device 62, the display 13, the CPU 60A, thestorage 60B, the memory 60C, and the communication I/Fs 66 to 68 isconnected to a bus 70. In the example illustrated in FIG. 5 , while onebus is illustrated as the bus 70 for convenience of illustration, aplurality of buses may be used. The bus 70 may be a serial bus or may bea parallel bus including a data bus, an address bus, a control bus, andthe like.

The memory 60C temporarily stores various information and is used as awork memory. Examples of the memory 60C include a RAM. However, thedisclosed technology is not limited thereto. A storage device of othertypes may be used. The storage 60B is a non-volatile storage device.Here, a flash memory is employed as an example of the storage 60B. Theflash memory is merely an example. Examples of the storage 60B includevarious non-volatile memories such as a magnetoresistive memory and/or aferroelectric memory instead of the flash memory or together with theflash memory. In addition, the non-volatile storage device may be anEEPROM, an HDD, and/or an SSD or the like. The storage 60B storesvarious programs for the management apparatus 11 (hereinafter, simplyreferred to as a “management apparatus program”). The CPU 60A controlsthe entire management apparatus 11 by reading out the managementapparatus program from the storage 60 B and executing the readmanagement apparatus program on the memory 60C.

The communication I/F 66 is, for example, a network interface. Thecommunication I/F 66 is communicably connected to the communication I/F34 of the surveillance camera 10 through a network and controls transferof various information with respect to the surveillance camera 10. Forexample, the communication I/F 66 requests the surveillance camera 10 totransmit the captured image and receives the captured image transmittedfrom the communication I/F 34 of the surveillance camera 10 in responseto the request for transmission of the captured image.

The communication I/Fs 67 and 68 are, for example, network interfaces.The communication I/F 67 is communicably connected to the driver 75through a network. The CPU 60 A controls a revolution operation of theyaw axis revolution mechanism 71 by controlling the motor 73 through thecommunication I/F 67 and the driver 75. The communication I/F 68 iscommunicably connected to the driver 76 through a network. The CPU 60 Acontrols a revolution operation of the pitch axis revolution mechanism72 by controlling the motor 74 through the communication I/F 68 and thedriver 76.

The reception device 62 includes, for example, a keyboard, a mouse, anda touch panel and receives various instructions from the user or thelike. The CPU 60 A acquires the various instructions received by thereception device 62 and operates in accordance with the acquiredinstructions.

The display 13 displays various information under control of the CPU60A. Examples of the various information displayed on the display 13include contents of the various instructions received by the receptiondevice 62 and the captured image received by the communication I/F 66.

The secondary storage device 14 stores various information under controlof the CPU 60A. Examples of the various information stored in thesecondary storage device 14 include the captured image received by thecommunication I/F 66.

In such a manner, the control device 60 performs a control of displayingthe captured image received by the communication I/F 66 on the display13 and a control of storing the captured image received by thecommunication I/F 66 in the secondary storage device 14. The capturedimage displayed on the display 13 is an example of an “image on which anadjustment result of a position of a target subject image is reflected”according to the embodiment of the disclosed technology. In addition,the captured image stored in the secondary storage device 14 is anexample of “image data” according to the embodiment of the disclosedtechnology.

Here, while the captured image is displayed on the display 13, and thecaptured image received by the communication I/F 66 is stored in thesecondary storage device 14, the disclosed technology is not limitedthereto. For example, any of the display of the captured image on thedisplay 13 and the storage of the captured image in the secondarystorage device 14 may be performed.

The surveillance camera 10 is provided with a function (hereinafter,referred to as a “tracking function”) of tracking the target subject. Ina case of tracking the target subject by the tracking function, it ispreferable that a position, in the captured image, of the target subjectimage showing the target subject (for example, a specific person)included in the imaging region is set to a predetermined position in thecaptured image in order to deal with movement of the target subject. Forexample, in a case where the position of the target subject image is setto a center position of the captured image, it is possible to deal witha change in position of the target subject image in various directions.

Therefore, in order to set the position of the target subject image inthe captured image to the predetermined position in the captured image,as illustrated in FIG. 6 as an example, a position adjustment program36A is stored in the storage 36, and the position adjustment program 36Ais executed by the CPU 37. Specifically, the CPU 37 functions as theimaging support device that supports imaging performed by thesurveillance camera 10 including the imaging element 25, by reading outthe position adjustment program 36A from the storage 36 and executingthe read position adjustment program 36 A on the memory 35. In such amanner, by functioning as the imaging support device, the CPU 37supports imaging under a state where the position of the target subjectimage of the surveillance camera 10 is set to the center position(hereinafter, referred to as an “image center position”) of the capturedimage. The image center position is an example of a “predeterminedposition” and a “specific position” according to the embodiment of thedisclosed technology.

In order to implement support of imaging under a state where theposition of the target subject image of the surveillance camera 10 isset to the specific position, the surveillance camera 10 comprises aposition adjustment portion 52. The position adjustment portion 52includes the revolution mechanism 16 and the shake correction portion 51and adjusts the position of the target subject image in the capturedimage. The CPU 37 supports imaging performed by the surveillance camera10 by controlling the position adjustment portion 52. The positionadjustment portion 52 is an example of a “position adjustment device”according to the embodiment of the disclosed technology.

In addition, the CPU 37 derives a movement amount required for movingthe position of the target subject image to the image center position bythe position adjustment portion 52 based on various information. Inaddition, the CPU 37 outputs the derived movement amount. By controllingthe position adjustment portion 52 based on the movement amount, theposition of the target subject image can be set to the image centerposition, and tracking of the target subject is supported.

The CPU 37 operates as an acquisition portion 37A, a derivation portion37B, a control portion 37C, an output portion 37D, and a determinationportion 37E by executing the position adjustment program 36A on thememory 35. The acquisition portion 37A is an example of an “acquisitionportion” according to the embodiment of the disclosed technology. Thederivation portion 37B is an example of a “derivation portion” accordingto the embodiment of the disclosed technology. The control portion 37Cis an example of a “control portion” according to the embodiment of thedisclosed technology. The output portion 37D is an example of an “outputportion” according to the embodiment of the disclosed technology. TheCPU 37 is an example of a “processor” according to the embodiment of thedisclosed technology. The memory 35 is an example of a “memory”according to the embodiment of the disclosed technology.

The determination portion 37E acquires the captured image from the imagememory 32 and performs image recognition of the target subject image onthe acquired captured image. The storage 36 stores an image recognitiondictionary 36B. The target subject image (for example, an image showinga specific object) as an image recognition target is registered in theimage recognition dictionary 36B. The determination portion 37Edetermines whether or not the target subject image is included in thecaptured image by referring to the image recognition dictionary 36B ofthe storage 36. In addition, in a case where the target subject image isincluded in the captured image, the determination portion 37E determineswhether or not the position of the target subject image is at the imagecenter position.

In a case where the determination portion 37E determines that theposition of the target subject image is at the image center position,the acquisition portion 37A acquires the captured image from the imagememory 32 and acquires a shift amount (hereinafter, referred to as an“in-image shift amount”) between the image center position and theposition of the target subject image by referring to the acquiredcaptured image. The acquisition portion 37A calculates the shift amountof pixel coordinates of the target subject image with respect to theimage center position (refer to FIG. 7 ). The shift amount of the pixelcoordinates is an example of an “in-image shift amount” according to theembodiment of the disclosed technology.

The acquisition portion 37 A acquires a focal length of the surveillancecamera 10. Specifically, the acquisition portion 37 A performssurveillance of a position of the zoom lens 15B2 on the optical axis OAand derives the focal length based on the surveillance result. Forexample, the derivation of the focal length is implemented using, by theacquisition portion 37A, a focal length derivation table in which thesurveillance result and the focal length are associated with each other,or a focal length derivation calculation expression that takes thesurveillance result as an independent variable and takes the focallength as a dependent variable.

The storage 36 stores sensitivity of the shake correction portion 51(hereinafter, simply referred to as the “sensitivity”) and a pixelinterval of pixels of the imaging element 25 (hereinafter, simplyreferred to as the “pixel interval”). The acquisition portion 37Aacquires the sensitivity from the storage 36. Here, the sensitivity is aproduct of a movement amount of the imaging region in thelight-receiving surface 25A per unit shake angle and a movable amount ofthe shake correction portion 51 necessary for moving the imaging regionin the light-receiving surface 25A by 1 degree. In addition, theacquisition portion 37A acquires the pixel interval from the storage 36.

The derivation portion 37B derives the movement amount required formoving the position of the target subject image to the image centerposition by the position adjustment portion 52. Specifically, thederivation portion 37B derives the movement amount based on the in-imageshift amount acquired by the acquisition portion 37A, the focal lengthacquired by the acquisition portion 37A, and the pixel interval. Thepixel interval is an example of “information related to a pixelinterval” according to the embodiment of the disclosed technology.

Here, while an example of a form of storing the pixel interval in thestorage 36 and acquiring the pixel interval from the storage 36 by theacquisition portion 37A is illustrated, the disclosed technology is notlimited thereto. The pixel interval may be derived from a size and thenumber of pixels of the captured image by the acquisition portion 37A orthe derivation portion 37B. In this case, information related to thesize and the number of pixels of the captured image is an example of the“information related to the pixel interval” according to the embodimentof the disclosed technology.

In addition, the acquisition portion 37A may acquire the informationrelated to the pixel interval from the imaging element 25. In this case,the derivation portion 37B may use the information related to the pixelinterval acquired by the acquisition portion 37A as the informationrelated to the pixel interval in deriving the movement amount.

The control portion 37C performs an adjustment control of adjusting theposition of the target subject image in the captured image by operatingthe position adjustment portion 52 based on the movement amount derivedby the derivation portion 37B. The control portion 37C performs theadjustment control of adjusting the position of the target subject imagein the captured image by operating the revolution mechanism 16 and theshake correction portion 51 included in the position adjustment portion52 based on the movement amount derived by the derivation portion 37B.

Here, while an example of a form of operating both of the revolutionmechanism 16 and the shake correction portion 51 of the positionadjustment portion 52 by the control portion 37C is illustrativelydescribed, the disclosed technology is not limited thereto. For example,the control portion 37C may operate the revolution mechanism 16 or theshake correction portion 51 based on the movement amount derived by thederivation portion 37B.

The control portion 37C performs the correction of the shake by theshake correction portion 51 and the adjustment control of adjusting theposition of the target subject image in the captured image in atime-division manner. Specifically, while the surveillance camera 10 isrevolving by the revolution mechanism 16, the control portion 37C causesthe shake correction portion 51 to correct the shake. While therevolution (hereinafter, referred to as “revolution at a time ofnon-adjustment”) of the surveillance camera 10 by the revolutionmechanism 16 is stopped, the control portion 37C performs the adjustmentcontrol. Here, the revolution of the surveillance camera 10 by therevolution mechanism 16 is broadly divided into the revolution at thetime of non-adjustment (revolution at a normal time) and the revolutionat a time of adjustment (revolution at other than the normal time). Therevolution at the time of adjustment refers to the revolution at thetime of adjustment of the position of the target subject image by therevolution mechanism 16 operated by performing the adjustment control.In other words, the time of adjustment refers to a timing different fromthe time of non-adjustment. In still other words, the time of adjustmentrefers to a timing at which the correction of the shake by the shakecorrection portion 51 is not performed. As the adjustment control, thecontrol portion 37C controls the revolution mechanism 16 and the shakecorrection portion 51 such that the position of the target subject imageis adjusted by the shake correction portion 51 after the position of thetarget subject image is adjusted by the revolution mechanism 16.

The output portion 37D outputs the movement amount derived by thederivation portion 37B. Specifically, the output portion 37D outputs themovement amount derived by the derivation portion 37B to the managementapparatus 11. The management apparatus 11 is an example of an “outside”according to the embodiment of the disclosed technology.

Next, an example of a method of deriving the movement amount derived bythe derivation portion 37B (hereinafter, simply referred to as the“movement amount”) will be described. The movement amount is decidedbased on a first movement amount required for adjusting the position ofthe target subject image by the revolution mechanism 16 and a secondmovement amount required for adjusting the position of the targetsubject image by the shake correction portion 51. Specifically, themovement amount derived by the derivation portion 37B is decided bycombining the first movement amount related to the revolution mechanism16 and the second movement amount related to the shake correctionportion 51.

As illustrated in FIG. 8 as an example, the first movement amount isdecided based on a value obtained by dividing a product of the in-imageshift amount and the pixel interval by the focal length. For example,the derivation portion 37B derives the first movement amount usingCalculation Expression (1) below. A pan tilt angle θ obtained byCalculation Expression (1) below is an example of a “first movementamount” according to the embodiment of the disclosed technology.

$\begin{matrix}{\theta = {\arctan( {p \times t/L} )}} & (1)\end{matrix}$

In Calculation Expression (1), θ denotes a pan tilt angle [deg], pdenotes the shift amount [pixel] of the pixel coordinates of theposition of the target subject image, t denotes the pixel interval[mm/pixel] of the imaging element, and L denotes the focal length [mm].

In addition, the derivation portion 37B derives the second movementamount. Specifically, the second movement amount is derived by thederivation portion 37B as the movement amount required for moving theshake correction element of the shake correction portion 51 to a centerposition from the current position. As illustrated in FIG. 9 as anexample, for a reason such as structural constraints on the surveillancecamera 10, a movable range is decided for the anti-vibration lens 15B1that is the shake correction element of the shake correction portion 51.Thus, as illustrated in FIG. 10 as an example, in a case where theanti-vibration lens 15B1 (hereinafter, referred to as the “shakecorrection element” without the reference numeral) is at a positionshifted from the center position in the movable range, the tracking ofthe target subject by the shake correction portion 51 may be restricteddepending on a correction amount required for correcting the shake. Thatis, in a case where the target subject moves in a direction outside themovable range of the shake correction element, it is difficult to trackthe target subject by the shake correction portion 51. Therefore, thederivation portion 37B derives the second movement amount that is themovement amount required for moving the shake correction element to thecenter position of the movable range (hereinafter, simply referred to asa “movable range center position”).

The derivation portion 37B derives a shake correction element movementamount required for moving the shake correction element to the movablerange center position from the current position based on a shakecorrection element shift amount and information related to thesensitivity of the shake correction portion 51 acquired by theacquisition portion 37A. The shift amount of the shake correctionelement is a distance between the movable range center position and thecurrent position of the shake correction element and is acquired by theacquisition portion 37A based on information output from the positiondetection sensor 39.

For example, the derivation portion 37B calculates the second movementamount using Calculation Expression (2) below. A pan tilt angle φobtained by Calculation Expression below is an example of a “secondmovement amount” according to the embodiment of the disclosedtechnology.

$\begin{matrix}{\varphi = {N/k}} & (2)\end{matrix}$

In Calculation Expression (2), N denotes the shake correction elementshift amount [mm], and k denotes the sensitivity [mm/deg].

In the present embodiment, while an example of a form of correcting theshake by moving the anti-vibration lens 15B1 is illustrativelydescribed, the disclosed technology is not limited thereto. For example,the shake may be corrected by moving the imaging element 25 in a planeparallel to the two-dimensional plane of the anti-vibration lens insteadof the anti-vibration lens 15B1. In this case, the shake can becorrected within a movable range of the imaging element 25. In addition,a shift amount from a center position of the movable range of theimaging element 25 is acquired by a position detection sensor (notillustrated) of the imaging element 25. In a case of using the imagingelement 25 instead of the anti-vibration lens 15B1 in correcting theshake, the imaging element 25 is an example of a “shake correctionelement” according to the embodiment of the disclosed technology. Inthis case, an imaging element moving type shake correction mechanism(not illustrated) that is an example of a “shake correction component”according to the embodiment of the disclosed technology corrects theshake by moving the imaging element in the direction perpendicular tothe optical axis of the imaging optical system by applying motive powergenerated by a driving source such as a motor (for example, a voice coilmotor) to the imaging element.

In addition, a mechanical shake correction mechanism (not illustrated)that is an example of the “shake correction component” and the “opticalshake correction mechanism” according to the embodiment of the disclosedtechnology may correct the shake by moving both of the anti-vibrationlens 15B1 and the imaging element 25 using motive power generated by adriving source such as a motor (for example, a voice coil motor). Inthis case, the shake can be corrected within the movable range of eachof the anti-vibration lens 15B1 and the imaging element 25. In thiscase, the anti-vibration lens 15B1 and the imaging element 25 are anexample of the “shake correction element” according to the embodiment ofthe disclosed technology.

The derivation portion 37B derives the movement amount by combining thefirst movement amount and the second movement amount. The movementamount derived by the derivation portion 37B is output by the outputportion 37D. As illustrated in FIG. 11 as an example, the movementamount output by the output portion 37D is displayed on the display 13of the management apparatus 11. Specifically, on the display 13, thecaptured image is displayed, and the pan tilt angle is graphed anddisplayed at a position adjacent to the captured image on a movementamount display screen as the movement amount. The user or the likecauses the surveillance camera 10 to revolve with reference to themovement amount displayed on the movement amount display screen.

In the example illustrated in FIG. 11 , while a case of visiblydisplaying the movement amount display screen is illustrated, audibledisplay such as output of sound by a sound reproducing device (notillustrated), permanent visible display such as output of a printedmaterial by a printer, or tactile display by a vibrator may be performedinstead of the visible display or together with the visible display.

Here, as illustrated in the upper part of FIG. 12 as an example, in aninitial state of the adjustment control performed by the control portion37C (hereinafter, simply referred to as the “initial state”), it isassumed that the position of the target subject image is shifted fromthe image center position. In addition, in the initial state, it isassumed that the current position of the shake correction element isshifted from the movable range center position. In this case, themovement amount derived by the derivation portion 37B is displayed onthe movement amount display screen.

The user or the like causes the surveillance camera 10 to revolve bycausing the revolution mechanism 16 to perform the revolution operationbased on the displayed movement amount (refer to the middle part of FIG.12 ). The position of the target subject image in the captured image isadjusted by operating the revolution mechanism 16.

After the position of the target subject image is adjusted by therevolution mechanism 16 (refer to the middle part of FIG. 12 ), theposition of the target subject image is adjusted by the shake correctionportion 51 (refer to the lower part of FIG. 12 ). Resolution of theadjustment of the position of the target subject image by the shakecorrection portion 51 is higher than resolution of the adjustment of theposition of the target subject image by the revolution mechanism 16.That is, in adjusting the position of the target subject image, the useror the like, first, performs coarse adjustment by the revolutionmechanism 16 and then, performs fine adjustment by the shake correctionportion 51. In a case where the position of the target subject image isadjusted by the shake correction portion 51, the position of the targetsubject image moves to the image center position. That is, an image onwhich the adjustment result of the position of the target subject imageis reflected is displayed as the captured image. In addition, in a casewhere the position of the target subject image is at the image centerposition, the shake correction element of the shake correction portion51 is positioned at the center of the movable range.

Next, actions of parts of the surveillance system 2 according to theembodiment of the disclosed technology will be described with referenceto FIG. 13 . FIG. 13 illustrates an example of a flow of positionadjustment processing executed by the CPU 37. The flow of positionadjustment processing illustrated in FIG. 13 is an example of an“imaging support method” according to the embodiment of the disclosedtechnology.

In the position adjustment processing illustrated in FIG. 13 , first, instep ST10, the determination portion 37E determines whether or not thetarget subject image is included in the captured image. In step ST10, ina case where the target subject image is included in the captured image,a positive determination is made, and the position adjustment processingtransitions to step ST12. In step ST10, in a case where the targetsubject image is not included in the captured image, a negativedetermination is made, and the position adjustment processingtransitions to step ST32.

In step ST12, the determination portion 37E determines whether or notthe position of the target subject image is at the image centerposition. In a case where the position of the target subject image is atthe image center position, a positive determination is made, and theposition adjustment processing transitions to step ST10. In step ST12,in a case where the position of the target subject image is not at theimage center position, a negative determination is made, and theposition adjustment processing transitions to step ST14.

In step ST14, the determination portion 37E determines whether or notposition adjustment by the revolution mechanism 16 is necessary formoving the position of the target subject image to the image centerposition. In step ST14, in a case where the position adjustment by therevolution mechanism 16 is not necessary, a negative determination ismade, and the position adjustment processing transitions to step ST20.In step ST14, in a case where the position adjustment by the revolutionmechanism 16 is necessary, a positive determination is made, and theposition adjustment processing transitions to ST16.

In step ST16, the acquisition portion 37A acquires the in-image shiftamount, the focal length, and the pixel interval. Then, the positionadjustment processing transitions to step ST18.

In step ST18, the derivation portion 37B derives the first movementamount based on the in-image shift amount, the focal length, and thepixel interval acquired in step ST18. Then, the position adjustmentprocessing transitions to step ST20.

In step ST20, the acquisition portion 37A determines whether or not theshake correction element is at the movable range center position. Instep ST20, in a case where the shake correction element is at themovable range center position, a positive determination is made, and theposition adjustment processing transitions to step ST26. In step ST20,in a case where the shake correction element is not at the centerposition of the movable range, a negative determination is made. Then,the position adjustment processing transitions to step ST22.

In step ST22, the acquisition portion 37A acquires the shake correctionelement shift amount and the sensitivity. Then, the position adjustmentprocessing transitions to step ST24.

In step ST24, the derivation portion 37B derives the second movementamount based on the shake correction element shift amount and thesensitivity. Then, the position adjustment processing transitions tostep ST26.

In step ST26, the derivation portion 37B derives the movement amountbased on the first movement amount and the second movement amount. Then,the position adjustment processing transitions to step ST28.

In step ST28, the output portion 37D outputs the movement amount. Then,the position adjustment processing transitions to step ST30.

In step ST30, the determination portion 37E determines whether or notthe shake correction element is at the center position of the movablerange. In step ST30, in a case where the shake correction element is notat the center position of the movable range, a negative determination ismade. Then, the position adjustment processing transitions to step ST30.In step ST30, in a case where the shake correction element is at themovable range center position, a positive determination is made, and theposition adjustment processing transitions to step ST32.

In step ST32, the determination portion 37E determines whether or not acondition (hereinafter, referred to as a “position adjustment processingfinish condition”) under which the position adjustment processing isfinished is satisfied. Examples of the position adjustment processingfinish condition include a condition that an instruction to finish theposition adjustment processing is received by the reception device 62.In step ST32, in a case where the position adjustment processing finishcondition is not satisfied, a negative determination is made, and theposition adjustment processing transitions to step ST10. In step ST32,in a case where the position adjustment processing finish condition issatisfied, a positive determination is made, and the position adjustmentprocessing is finished.

As described above, in the surveillance camera 10, the target subjectmay be tracked in capturing of the imaging region including the targetsubject. In this case, the position of the target subject image in thecaptured image may be required to be set to the specific position inorder to deal with the movement of the target subject.

Therefore, in the surveillance camera 10, the movement amount requiredfor moving the position of the target subject image in the capturedimage to the specific position by the position adjustment portion 52 isderived based on the in-image shift amount, the focal length, and theinformation related to the pixel interval. In addition, in thesurveillance camera 10, the derived movement amount is output.Accordingly, in the surveillance camera 10, tracking of the targetsubject in a case of capturing the imaging region including the targetsubject can be supported.

In addition, in the surveillance camera 10, the revolution mechanism 16and the shake correction portion 51 are used as the position adjustmentportion 52. Accordingly, by using the already provided revolutionmechanism and the shake correction portion 51 as the position adjustmentportion 52, a configuration is simplified compared to a case ofseparately providing the position adjustment portion 52.

In addition, in the surveillance camera 10, the movement amount isdecided based on the first movement amount required for adjusting theposition of the target subject image by the revolution mechanism 16 andthe second movement amount required for adjusting the position of thetarget subject image by the shake correction portion 51. Accordingly,the movement amount appropriate for each of the revolution mechanism 16and the shake correction portion 51 is set compared to a case where themovement amounts of the revolution mechanism 16 and the shake correctionportion 51 are predetermined values.

In addition, in the surveillance camera 10, the shake correction portion51 includes the shake correction element that is at least one of theanti-vibration lens 15B1 or the imaging element 25. In a case where theposition of the target subject image is at the specific position, theshake correction element is positioned at the center of the movablerange of the shake correction element. Accordingly, a wide trackablerange of the target subject image can be secured compared to a casewhere the shake correction element is at a location other than thecenter after the position of the target subject image is set to thespecific position.

In addition, in the surveillance camera 10, the sensitivity of the shakecorrection portion 51 is acquired, and the movement amount required formoving the current position of the shake correction element to thecenter position of the movable range is derived as the second movementamount based on the shift amount of the shake correction element and thesensitivity. Accordingly, since the movement amount of the targetsubject image is decided based on the shake correction element shiftamount and the sensitivity, the movement amount of the target subjectimage is accurately obtained compared to a case where the movementamount of the position of the target subject image for the shakecorrection portion 51 is an invariable value.

In addition, in the surveillance camera 10, the first movement amount isdecided as a value obtained by dividing the product of the in-imageshift amount and the pixel interval by the focal length. Accordingly,since the movement amount of the target subject image is decided basedon the in-image shift amount, the pixel interval, and the focal length,the movement amount of the target subject image is accurately obtainedcompared to a case where the movement amount of the position of thetarget subject image for the revolution mechanism 16 is an invariablevalue.

In addition, in the surveillance camera 10, the movement amount isdecided by combining the first movement amount and the second movementamount. Accordingly, since the movement amount is derived as a valueobtained by combining the first movement amount and the second movementamount, subsequent processing such as display on the display portion isfacilitated compared to a case of separately deriving the first movementamount and the second movement amount.

In addition, in the surveillance camera 10, the shake correction by theshake correction portion 51 and the adjustment control are performed ina time-division manner. Accordingly, an effect of one of the correctionof the shake by the shake correction portion 51 and the adjustmentcontrol on the other can be suppressed compared to a case where thecorrection of the shake by the shake correction portion 51 and theadjustment control are performed in parallel.

In addition, in the surveillance camera 10, the correction of the shakeby the shake correction portion 51 is performed while the revolution bythe revolution mechanism 16 is performed, and the adjustment control isperformed while the revolution by the revolution mechanism 16 isstopped. Accordingly, both of accuracy of the correction of the shake bythe shake correction portion 51 and accuracy of the adjustment controlcan be increased compared to a case where the correction of the shake bythe shake correction portion 51 and the adjustment control are performedin parallel while the surveillance camera 10 is revolving and while therevolution of the surveillance camera 10 by the revolution mechanism 16is stopped.

In addition, in the surveillance camera 10, after the position of thetarget subject image is adjusted by the revolution mechanism 16, theposition of the target subject image is adjusted within a narrower rangeby the shake correction portion 51. Accordingly, accuracy of theadjustment of the position of the target subject image is improvedcompared to a case of adjusting the position of the target subject imageby only the revolution mechanism.

In addition, in the surveillance camera 10, the revolution mechanism 16is a 2-axis revolution mechanism, and the shake correction portion 51 isat least one of an optical shake correction mechanism or an electronicshake correction portion. Accordingly, by combining the 2-axisrevolution mechanism and at least one of the optical shake correctionmechanism or the electronic shake correction portion, the movementamount required for adjusting the position of the target subject imagecan be secured for each direction of two axes.

In addition, in the surveillance camera 10, the optical shake correctionmechanism is at least one of a lens moving type shake correctionmechanism or an imaging element moving type shake correction mechanism.Accordingly, by at least one of the lens moving type shake correctionmechanism or the imaging element moving type shake correction mechanism,the movement amount required for adjusting the position of the targetsubject image can be secured for each direction of two axes.

In addition, in the surveillance camera 10, the information related tothe pixel interval of the pixels of the imaging element 25 is acquiredby the acquisition portion 37A. Accordingly, even in a case where theinformation related to the pixel interval is updated, the movementamount required for moving the position of the target subject image tothe specific position can be derived using the most recent informationrelated to the pixel interval.

In addition, in the surveillance camera 10, the derived movement amountis output to the outside. Accordingly, the user or the like can perceivethe movement amount required for moving the position of the targetsubject image to the specific position.

In the embodiment, while an example in which the movement amount outputto the outside is displayed on the display 13 of the managementapparatus 11 is illustrated, the disclosed technology is not limitedthereto. As illustrated in FIG. 14 as an example, the movement amountoutput from the output portion 37D may be displayed on the display 43Bprovided in the surveillance camera 10. The display 43B is an example ofthe “outside” according to the embodiment of the disclosed technology.

Specifically, as illustrated in FIG. 15 as an example, the movementamount derived by the derivation portion 37B is output from the outputportion 37D. The output portion 37D outputs the movement amount to adisplay portion 53. Examples of the display portion 53 include thedisplay 13 and/or the display 43B. Accordingly, the user or the like ofthe surveillance camera 10 can perceive the movement amount required formoving the position of the target subject image to the image centerposition.

In the example illustrated in FIG. 14 , while a case of visiblydisplaying the movement amount display screen is illustrated, audibledisplay such as output of sound by a sound reproducing device (notillustrated), permanent visible display such as output of a printedmaterial by a printer, or tactile display by a vibrator may be performedinstead of the visible display or together with the visible display.

In addition, in the embodiment, while an example in which the revolutionby the revolution mechanism 16 is performed by the user or the like ofthe surveillance camera 10 is illustrated, the disclosed technology isnot limited thereto. As illustrated in FIG. 16 as an example, thesurveillance camera 10 and the revolution mechanism 16 may becommunicably connected through the communication line 12. In the exampleof the present form, the movement amount required for moving theposition of the target subject image to the image center position isoutput from the control portion 37C in the surveillance camera 10 to therevolution mechanism 16. The revolution mechanism 16 causes thesurveillance camera 10 to revolve by the movement amount input from thecontrol portion 37C.

Second Embodiment

In the first embodiment, a case where the target subject does not moveis described. In a second embodiment, a case where the target subjectmoves will be described. In the second embodiment, the same constituentsas in the first embodiment will be designated by the same referencenumerals and will not be described. Hereinafter, parts different fromthe first embodiment will be described.

In imaging in the surveillance camera 10 according to the secondembodiment, as illustrated in FIG. 17 as an example, the position of thetarget subject image in the captured image changes over time by themovement of the target subject. That is, the in-image shift amountbetween the image center position and the position of the target subjectimage changes. Thus, the movement amount required for moving theposition of the target subject image to the image center position alsoneeds to be obtained as a movement amount in which the movement of thetarget subject is considered.

Specifically, as illustrated in FIG. 18 as an example, the determinationportion 37E determines whether or not the position of the target subjectimage is changing in the captured image. The acquisition portion 37Aacquires a movement velocity of the target subject image by calculatinga movement amount per unit time for the target subject image between theframes of the captured image stored in the image memory 32. In a casewhere the position of the target subject image is changing in thecaptured image, the derivation portion 37B derives the movement amountrequired for setting the position of the target subject image to theimage center position based on the movement velocity.

The determination portion 37E decomposes the movement velocity of thetarget subject image into a plurality of velocities in a verticaldirection and a horizontal direction in the captured image. The verticaldirection and the horizontal direction in the captured image are anexample of a “plurality of different directions” according to theembodiment of the disclosed technology. For the movement velocitiesobtained by decomposing the movement velocity of the target subjectimage into a component of the vertical direction and a component of thehorizontal direction of the captured image, the determination portion37E obtains a speed of the component of each direction. Furthermore, thedetermination portion 37E compares the speeds of the component of thehorizontal direction and the component of the vertical direction of thecaptured image. In addition, the determination portion 37E determineswhether or not a component of a direction of a higher speed heads towardthe image center position side. The derivation portion 37B determineswhether or not to output the derived movement amount based on thedetermination result as to whether or not the component of the directionof the higher speed is a direction of heading toward the image centerposition side.

As illustrated in FIG. 19 as an example, in a case where the position ofthe target subject image is changing in a rightward direction of headingtoward the image center position side in the captured image, the speedof the component of the horizontal direction is greater than thecomponent of the vertical direction of the captured image. The componenthaving the higher speed (in the example illustrated in FIG. 19 , thecomponent of the horizontal direction of the captured image) is acomponent of the direction of heading toward the image center positionside. Consequently, the derivation portion 37 B does not output themovement amount in the horizontal direction (pan direction) of thecaptured image. Accordingly, in the initial state, the movement amountin only the vertical direction (tilt direction) of the captured image isdisplayed on the movement amount display screen (refer to the upper partof FIG. 19 ). After position adjustment by the position adjustmentportion 52 is completed, the position of the target subject image in thecaptured image changes in a direction of approaching the image centerposition (refer to the lower part of FIG. 19 ).

Meanwhile, as illustrated in FIG. 20 , in a case where the position ofthe target subject image is changing in a leftward direction ofseparating from the image center position in the captured image, thecomponent having the higher speed is a direction of separating from theimage center position. In this case, the derivation portion 37B outputsthe movement amounts in the horizontal direction (pan direction) andalso the vertical direction (tilt direction) of the captured image.Accordingly, in the initial state, the movement amounts (pan tiltangles) in the horizontal direction (pan direction) and the verticaldirection (tilt direction) of the captured image are displayed on themovement amount display screen (refer to the upper part of FIG. 20 ).After the position adjustment by the position adjustment portion 52 iscompleted, the position of the target subject image in the capturedimage is at the specific position (refer to the lower part of FIG. 20 ).

Next, actions of parts of the surveillance system 2 according to theembodiment of the disclosed technology will be described with referenceto FIG. 21A and FIG. 21B. FIG. 21A and FIG. 21B illustrate an example ofthe flow of position adjustment processing executed by the CPU 37. Theflow of position adjustment processing illustrated in FIG. 21A and FIG.21B is an example of the “imaging support method” according to theembodiment of the disclosed technology.

In the position adjustment processing illustrated in FIG. 21A, first, instep ST10, the determination portion 37E determines whether or not thetarget subject image is included in the captured image. In step ST10, ina case where the target subject image is included in the captured image,a positive determination is made, and the position adjustment processingtransitions to step ST34. In step ST10, in a case where the targetsubject image is not included in the captured image, a negativedetermination is made, and the position adjustment processingtransitions to step ST32.

In step ST34, the determination portion 37E executes velocitydetermination processing illustrated in FIG. 21B as an example. In thevelocity determination processing illustrated in FIG. 21B, first, instep ST40, the determination portion 37E determines whether or not theposition of the target subject image is moving in the captured image. Instep ST40, in a case where the position of the target subject image ismoving, a positive determination is made, and the velocity determinationprocessing transitions to step ST42. In the determination in step ST40,in a case where the position of the target subject image is not moving,a negative determination is made, and the velocity determinationprocessing transitions to step ST14 of the position adjustmentprocessing illustrated in FIG. 21A.

In step ST42, the movement velocity of the target subject is decomposedinto two different directions by the determination portion 37E. Then,the velocity determination processing transitions to step ST44.

In step ST44, the determination portion 37E compares the speed of thecomponent of each direction decomposed in step ST42. Then, the velocitydetermination processing transitions to step ST46.

In step ST46, the determination portion 37E determines whether or notthe component having the higher speed out of the directions compared instep ST44 is the direction of heading toward the image center position.In step ST46, in a case where the component having the higher speed isthe direction of heading toward the image center position, a positivedetermination is made, and the velocity determination processingtransitions to step ST48. In step ST46, in a case where the componenthaving the higher speed is not the direction of heading toward the imagecenter position, a negative determination is made, and the velocitydetermination processing transitions to step ST14 of the positionadjustment processing illustrated in FIG. 21A.

In step ST48, the derivation portion 37 B controls the output portion37D such that the movement amount of the component having the higherspeed is not output. Then, the velocity determination processingtransitions to step ST12 of the position adjustment processingillustrated in FIG. 21A.

As described above, in the surveillance camera 10 according to thesecond embodiment, even while the target subject is moving, the positionadjustment in which the movement velocity of the moving target subjectis considered can be performed.

In addition, in the surveillance camera 10 according to the secondembodiment, even while the target subject is moving, high-accuracyposition adjustment can be implemented compared to a case where themovement velocity is a velocity in only a single direction.

In each of the embodiments, while an example of a form in which themovement amount obtained by combining the first movement amount and thesecond movement amount is displayed on the display 13 is illustrativelydescribed, the disclosed technology is not limited thereto. For example,the first movement amount and the second movement amount may bedisplayed by separate indicators.

In addition, in each of the embodiments, while an example in which thespecific position and the predetermined position are the image centerposition is illustratively described, the disclosed technology is notlimited thereto. The specific position and the predetermined positionmay be set to any position (for example, one of four corners in thecaptured image) in the captured image by the user or the like.

In addition, in each of the embodiments, while a device including anASIC and an FPGA is illustrated, the disclosed technology is not limitedthereto. Various processing may be implemented by a softwareconfiguration using a computer.

In this case, for example, as illustrated in FIG. 22 , the computer 19is incorporated in the surveillance camera 10. The position adjustmentprogram 36A causing the computer 19 to execute the position adjustmentprocessing according to the embodiments is stored in a storage medium100 that is a non-transitory storage medium. Examples of the storagemedium 100 include any portable storage medium such as an SSD or a USBmemory.

The computer 19 comprises the CPU 37, the storage 36, and the memory 35.The storage 36 is a non-volatile storage device such as an EEPROM, andthe memory 35 is a volatile storage device such as a RAM. The positionadjustment program 36A stored in the storage medium 100 is installed onthe computer 19. The CPU 37 executes the position adjustment processingin accordance with the position adjustment program 36A.

The position adjustment program 36 A may be stored in the storage 36instead of the storage medium 100. In this case, the CPU 37 reads outthe position adjustment program 36A from the storage 36 and executes theread position adjustment program 36A on the memory 35. In such a manner,the position adjustment processing is implemented by executing theposition adjustment program 36A by the CPU 37.

In addition, the position adjustment program 36A may be stored in astorage portion of another computer, a server apparatus, or the likeconnected to the computer 19 through a communication network (notillustrated), and the position adjustment program 36A may be downloadedand installed on the computer 19 in response to a request of thesurveillance camera 10.

In the storage portion of the other computer, the server apparatus, orthe like connected to the computer 19 or in the storage 36, the entireposition adjustment program 36A does not need to be stored, and a partof the position adjustment program 36A may be stored.

In the example illustrated in FIG. 22 , while an example of an aspect ofincorporating the computer 19 in the surveillance camera 10 isillustrated, the disclosed technology is not limited thereto. Forexample, the computer 19 may be provided outside the surveillance camera10.

In the example illustrated in FIG. 22 , while the CPU 37 is a singleCPU, the CPU 37 may include a plurality of CPUs. In addition, a GPU maybe applied instead of the CPU 37.

In the example illustrated in FIG. 22 , while the computer 19 isillustrated, the disclosed technology is not limited thereto. A deviceincluding an ASIC, an FPGA, and/or a PLD may be applied instead of thecomputer 19. In addition, a combination of a hardware configuration anda software configuration may be used instead of the computer 19.

Various processors illustrated below can be used as a hardware resourcefor executing the position adjustment processing described in each ofthe embodiments. Examples of the processors include a CPU that is ageneral-purpose processor functioning as the hardware resource forexecuting the position adjustment processing by executing software, thatis, the program. In addition, examples of the processors include adedicated electric circuit such as an FPGA, a PLD, or an ASIC that is aprocessor having a circuit configuration dedicatedly designed to executespecific processing. Any of the processors incorporates or is connectedto a memory, and any of the processors executes the position adjustmentprocessing using the memory.

The hardware resource for executing the position adjustment processingmay be configured with one of those various processors or may beconfigured with a combination of two or more processors of the same typeor different types (for example, a combination of a plurality of FPGAsor a combination of a CPU and an FPGA). In addition, the hardwareresource for executing the position adjustment processing may be oneprocessor.

Examples of a configuration with one processor include, first, a form inwhich one processor is configured with a combination of one or more CPUsand software, and the processor functions as the hardware resource forexecuting the position adjustment processing. Second, as represented byan SoC or the like, a form of using a processor that implements, by oneIC chip, functions of the entire system including a plurality ofhardware resources for executing the position adjustment processing isincluded. In such a manner, the position adjustment processing isimplemented using one or more of the various processors as the hardwareresource.

Furthermore, more specifically, an electric circuit in which circuitelements such as semiconductor elements are combined can be used as ahardware structure of those various processors. In addition, theposition adjustment processing is merely an example. Accordingly,unnecessary steps may be deleted, new steps may be added, or aprocessing order may be rearranged without departing from the gist ofthe disclosed technology.

In addition, while the surveillance camera 10 is illustrated in theexample illustrated in FIG. 1 , the disclosed technology is not limitedthereto. That is, the disclosed technology can be applied to variouselectronic apparatuses (for example, a lens-interchangeable camera, afixed lens camera, a smart device, a personal computer, and/or awearable terminal apparatus or the like) incorporating the imagingapparatus. Even with these electronic apparatuses, the same actions andeffects as the surveillance camera 10 are obtained.

In addition, while the display 43B is illustrated in each of theembodiments, the disclosed technology is not limited thereto. Forexample, a separate display that is connected to the imaging apparatusmay be used as the “display portion” according to the embodiment of thedisclosed technology.

Above described contents and illustrated contents are detaileddescription for parts according to the embodiment of the disclosedtechnology and are merely an example of the disclosed technology. Forexample, description related to the above configurations, functions,actions, and effects is description related to an example ofconfigurations, functions, actions, and effects of the parts accordingto the embodiment of the disclosed technology. Thus, unnecessary partsmay be removed, new elements may be added, or parts may be replaced inthe above described contents and the illustrated contents withoutdeparting from the gist of the disclosed technology. In addition,particularly, description related to common technical knowledge or thelike that does not need to be described in terms of embodying thedisclosed technology is omitted in the above described contents and theillustrated contents in order to avoid complication and facilitateunderstanding of the parts according to the embodiment of the disclosedtechnology.

In the present specification, “A and/or B” has the same meaning as “atleast one of A or B”. This means that “A and/or B” may be only A, onlyB, or a combination of A and B. In addition, in the presentspecification, the same approach as “A and/or B” is applied to a casewhere three or more matters are represented by connecting the matterswith “and/or”.

All documents, patent applications, and technical standards disclosed inthe present specification are incorporated in the present specificationby reference to the same extent as in a case where each of thedocuments, patent applications, technical standards are specifically andindividually indicated to be incorporated by reference.

The following appendix is further disclosed with respect to theembodiments.

(Appendix)

An information processing apparatus including a processor, and a memorythat is incorporated in or connected to the processor, in which theprocessor is configured to acquire an in-image shift amount between apredetermined position in a captured image obtained by capturing animaging region including a target subject by an imaging element and aposition of a target subject image showing the target subject, and afocal length of an imaging apparatus, derive a movement amount requiredfor moving the position of the target subject image to a specificposition by a position adjustment portion which adjusts the position ofthe target subject image in the captured image, based on the in-imageshift amount acquired by the acquisition portion, the focal lengthacquired by the acquisition portion, and information related to a pixelinterval of pixels in the imaging element, and output the movementamount derived by the derivation portion.

What is claimed is:
 1. An imaging support device comprising: aprocessor; and a memory that is incorporated in or connected to theprocessor, the imaging support device supporting imaging performed by animaging apparatus including an imaging element, wherein the processor isconfigured to acquire an in-image shift amount between a predeterminedposition in a captured image obtained by capturing an imaging regionincluding a target subject by the imaging element and a position of atarget subject image showing the target subject, and a focal length ofthe imaging apparatus, derive a movement amount required for moving theposition of the target subject image to a specific position by aposition adjustment device which adjusts the position of the targetsubject image in the captured image, based on the acquired in-imageshift amount, the acquired focal length, and information related to apixel interval of pixels in the imaging element, and output the derivedmovement amount.
 2. The imaging support device according to claim 1,wherein the position adjustment device includes a revolution mechanismto which the imaging apparatus is attached and that enables the imagingapparatus to revolve, and a shake correction component that corrects ashake which occurs due to a vibration exerted on the imaging apparatus,and the processor is further configured to perform adjustment controlfor adjusting the position of the target subject image in the capturedimage by operating at least one of the revolution mechanism or the shakecorrection component based on the movement amount.
 3. The imagingsupport device according to claim 2, wherein the movement amount isdecided based on a first movement amount required for adjusting theposition of the target subject image by the revolution mechanism and asecond movement amount required for adjusting the position of the targetsubject image by the shake correction component.
 4. The imaging supportdevice according to claim 3, wherein the first movement amount isdecided based on a value obtained by dividing a product of the in-imageshift amount and the pixel interval by the focal length.
 5. The imagingsupport device according to claim 3, wherein the movement amount isobtained by combining the first movement amount and the second movementamount derived by the processor.
 6. The imaging support device accordingto claim 2, wherein the shake correction component includes a shakecorrection element that is at least one of a lens for correcting theshake by moving in accordance with the vibration or the imaging element,and at the specific position, the shake correction element is positionedat a center of a movable range of the shake correction element.
 7. Theimaging support device according to claim 6, wherein the processorconfigured to further acquire sensitivity of the shake correctioncomponent, and derive, based on a shake correction element shift amountbetween a center position of the movable range and a current position ofthe shake correction element, and the acquired sensitivity, a shakecorrection element movement amount required for moving the currentposition to the center position as a second movement amount required foradjusting the position of the target subject image by the shakecorrection component.
 8. The imaging support device according to claim2, wherein the processor is configured to perform correction of theshake by the shake correction component and the adjustment control in atime-division manner.
 9. The imaging support device according to claim8, wherein the processor is configured to cause the shake correctioncomponent to perform the correction of the shake while the imagingapparatus is revolving by the revolution mechanism, and perform theadjustment control while the revolution of the imaging apparatus by therevolution mechanism is stopped.
 10. The imaging support deviceaccording to claim 8, wherein the adjustment control is control foradjusting the position of the target subject image by the shakecorrection component after the position of the target subject image isadjusted by the revolution mechanism.
 11. The imaging support deviceaccording to claim 2, wherein the revolution mechanism is a 2-axisrevolution mechanism that enables the imaging apparatus to revolve in afirst direction and a second direction which intersects with the firstdirection, and the shake correction component is at least one of anoptical shake correction mechanism or an electronic shake correctioncomponent.
 12. The imaging support device according to claim 11, whereinthe optical shake correction mechanism is at least one of a lens movingtype shake correction mechanism or an imaging element moving type shakecorrection mechanism.
 13. The imaging support device according to claim1, wherein the processor is configured to further acquire theinformation related to the pixel interval.
 14. The imaging supportdevice according to claim 1, wherein the processor is configured tooutput the movement amount to an exterior.
 15. The imaging supportdevice according to claim 1, wherein the movement amount is decidedbased on a movement velocity of the target subject in a case where thetarget subject is moving.
 16. The imaging support device according toclaim 15, wherein the movement velocity includes a plurality ofvelocities obtained by decomposing the movement velocity into aplurality of different directions.
 17. An imaging apparatus comprising:the imaging support device according to claim 1; and the imagingelement, wherein the imaging support device supports imaging for theimaging element.
 18. An imaging system comprising: the imaging apparatusaccording to claim 17; and a control device that performs at least oneof control for displaying an image on which an adjustment result of theposition of the target subject image is reflected based on the movementamount derived by the processor on a display, or control for storingimage data indicating the image on which the adjustment result isreflected in a storage device.
 19. An imaging support system comprising:the imaging support device according to claim 1; and the positionadjustment device, wherein the processor included in the imaging supportdevice derives the movement amount.
 20. An imaging system comprising: animaging element; and an imaging support device that includes a processorand a memory which is incorporated in or connected to the processor, andsupports imaging performed by an imaging apparatus including the imagingelement, wherein the processor is configured to acquire an in-imageshift amount between a predetermined position in a captured imageobtained by capturing an imaging region including a target subject bythe imaging element and a position of a target subject image showing thetarget subject, and a focal length of the imaging apparatus, derive amovement amount required for moving the position of the target subjectimage to a specific position by a position adjustment device whichadjusts the position of the target subject image in the captured image,based on the acquired in-image shift amount, the acquired focal length,and information related to a pixel interval of pixels in the imagingelement, and output the derived movement amount, the position adjustmentdevice includes a revolution mechanism to which the imaging apparatus isattached and that enables the imaging apparatus to revolve, and a shakecorrection component that corrects a shake which occurs due to avibration exerted on the imaging apparatus, and the processor isconfigured to perform adjustment control for adjusting the position ofthe target subject image in the captured image by operating at least oneof the revolution mechanism or the shake correction component based onthe movement amount.
 21. An imaging support method of supporting imagingperformed by an imaging apparatus including an imaging element, theimaging support method comprising: acquiring an in-image shift amountbetween a predetermined position in a captured image obtained bycapturing an imaging region including a target subject by the imagingelement and a position of a target subject image showing the targetsubject, and a focal length; deriving a movement amount required formoving the position of the target subject image to a specific positionby a position adjustment device which adjusts the position of the targetsubject image in the captured image, based on the acquired in-imageshift amount, the acquired focal length, and a pixel interval of pixelsof the imaging element; and outputting the derived movement amount. 22.A non-transitory computer-readable storage medium storing a programexecutable by a computer to perform a process of supporting imagingperformed by an imaging apparatus including an imaging element, theprocess comprising: acquiring an in-image shift amount between apredetermined position in a captured image obtained by capturing animaging region including a target subject by the imaging element and aposition of a target subject image showing the target subject, and afocal length; deriving a movement amount required for moving theposition of the target subject image to a specific position by aposition adjustment device which adjusts the position of the targetsubject image in the captured image, based on the acquired in-imageshift amount, the acquired focal length, and a pixel interval of pixelsof the imaging element; and outputting the derived movement amount.